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Abstract 


The  down-sizing  of  the  Department  of  Defense  cuts  across  all  branches  of  the 
military.  Measurement  of  the  effectiveness  of  the  leaner  fighting  force  is  essential  to 
allow  decision  makers  to  spend  the  shrinking  military  budget  in  the  most  cost-effective 
manner.  The  medical  departments  of  the  three  services  have  been  included  in  this 
down-sizing;  however,  their  missions  have  not  been  down-sized.  New  methods  and 
systems  for  die  delivery  of  healthcare  to  military  beneficiaries  are  being  implemented. 
The  evaluation  of  healthcare  provided  under  the  new  delivery  system  is  essential  since 
the  budget  for  healthcare  is  also  being  down-sized.  The  quality  and  effectiveness  of  the 
care  provided  must  continue  at  high  levels  while  costs  are  kept  as  low  as  possible.  The 
selection  of  utilization  management  (UM)  guidelines  for  use  in  Department  of  Defense 
medical  treatment  facilities  has  far  reaching  consequences.  These  guidelines  are  used  to 
evaluate  the  care  provided  not  only  in  the  military  hospitals  and  clinics,  but  also  the  care 
provided  by  the  contracted  provider  networks.  The  selection  of  UM  guidelines  should 
be  based  on  criteria  selected  by  individuals  with  experience  and  e7q)ertise  in  utilization 
management.  UM  guidelines  should  not  be  selected  based  only  on  by  die  dollar  amount 
attached  to  the  contract  The  rework  diat  the  selection  of  the  wrong  set  of  guidelines 
could  cost  would  negate  any  perceived  savings  on  the  up-front  cost  of  the  contract, 
regardless  of  the  number  of  SMEs  (12  vs  7)  included  in  the  analysis  or  number  of  consensus 
criteria  (18  vs  16)  M&R  consistently  and  substantially  was  found  the  siqierior  tool. 
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1.  INTRODUCTION 


The  American  appetite  for  the  latest  and  greatest  in  healthcare,  no  matter  the 
costs,  has  helped  fuel  a  rapid  upward  spiral  of  healthcare  costs.  Healthcare  spending 
across  the  nation  has  seen  an  unprecedented  cost  explosion  over  the  past  several 
decades.  The  response  to  this  fiscal  emergency  has  been  the  implementation  of 
programs  designed  to  slow  the  expenditure  of  scarce  healthcare  dollars.  The  American 
healthcare  industry  has  seen  ever-increasing  constraints  placed  on  it  by  third  party 
payers  as  they  try  to  slow  their  escalating  costs.  The  greatest  driving  force  behind  the 
constraints  has  been  the  upward  spiral  of  healthcare  costs  associated  with  the 
healthcare  technology  explosion.  Changes  in  healthcare  payment  practices  have 
brought  to  the  forefront  the  need  to  contain  costs  while  providing  quality  care. 

The  impact  of  cost-containment  measures  must  be  monitored  to  observe  their 
impact  on  the  quality  and  outcome  of  healthcare  services  (Dring,  et  el.  1994). 
Escalating  medical  costs  and  cost  containment  measures  have  created  an  increased 
interest  in  studies  aimed  at  improving  efficiency  in  care  delivery  without  affecting  the 
quality  of  care  delivered  (MacKoul,  et  el.  19^).  Everyone  involved  in  the  purchase  of 
healthcare  is  concerned  about  the  quality  of  care  provided.  Today’s  healthcare 
consumer  is  better  informed  than  at  any  time  in  history,  and  is  concerned  about  not 
only  the  quality  of  care  received,  but  the  costs  involved.  Healthcare  costs  are 
measured  in  more  than  just  direct  dollars,  they  include  costs  related  to  waiting  times, 
either  on  the  telephone  or  in  the  provider’s  waiting  room.  Purchasers  in  both  the 
private  and  public-sector  want  both  lower  costs  and  quality  care  (Gosfield  1996). 
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The  number  of  variables  involved  make  it  difficult  to  evaluate  healthcare. 


Structure,  process,  and  outcome  are  not  always  evaluated  when  a  determination  of  the 
quality  of  care  is  made.  If  quality  is  defined  as  meeting  or  exceeding  expectations, 
how  those  expectations  are  established  has  a  pivotal  impact  on  judgments  of  quality 
(Hungate  1994).  It  has  been  said  that  the  quality  of  healthcare  is  measured  through  the 
eyes  of  the  consumer.  If  a  consumer  feels  good  about  the  healthcare  interaction, 
he/she  will  likely  perceive  quality  care  has  been  delivered.  Consumer  perception  can 
be  swayed  by  the  personalities  of  the  providers  encountered,  their  actions,  the  physical 
environment  of  the  setting,  and  feeling  better  as  a  result  of  the  encounter.  Healthcare 
providers  have  their  own  perception  of  the  quality  of  the  care  they  provide.  A 
provider’s  perception  of  the  care  delivered  is  usually  based  on  the  outcome;  did  the 
patient’s  condition  improve.  Rarely  will  a  provider  examine  aspects  of  care  such  as: 
appropriateness  of  care,  length  of  stay,  cost,  use  of  appropriate  referrals,  and  the 
setting  used  for  care  provision.  A  providers  evaluation  of  care  focuses  on  flie  technical 
quality  of  care.  Very  little  is  done  to  address  the  perceived  quality  of  care.  The 
perceived’  quality  of  care  issues  are  usually  addressed  through  the  patient 
representative’s  office. 

Efforts  to  reduce  lengths  of  stay  have  taken  the  form  of  managed  care 
programs,  which  attempt  to  reduce  hospital  stays  through  the  implementation  of 
clinical  guidelines  and  critical  pathways  for  specific  procedures  and  diagnoses. 
Available  information  suggests  hospital  lengths  of  stay  are  decreasing  over  time,  and 
there  is  an  increasing  awareness  that  patients  are  generally  better  off  without  extended 
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hospital  stays.  The  widespread  interest  in  length  of  stay  reduction  has  generated  little 
information  concerning  the  limits  of  hospital  stay  reduction  for  specific  procedures 
and  diagnoses.  Related  to  this  problem  is  a  lack  of  information  concerning  how 
quality  of  care  is  affected  by  decreases  in  lengths  of  stay.  Currently,  there  are  no 
reliable  estimates  of  optimum  hospital  stays  by  procedure  or  diagnosis.  In  most 
instances,  the  shortest  hospital  stay  is  implicitly  assumed  to  be  the  most  appropriate 
length  of  stay  (Westert  1995). 

To  a  physician,  health  status  is  a  more  appropriate  outcome  measure  than 
length  of  hospital  stay.  To  determine  cost-effectiveness,  it  is  essential  to  know 
whether  the  cost  is  justified  by  the  benefit  to  the  patient  (Winkelaar  1995).  To 
decrease  length  of  stay,  institutions  have  developed  techniques  to  streamline  patient 
care  while  maintaining  quality.  These  techniques  include  case  management,  managed 
care,  and  the  implementation  of  standardized  patient  care  plans  or  critical  pathways 
(Lombness  1994).  The  impetus  behind  managed  healthcare  is  cost  containment 
without  a  loss  of  quality  (Gustin,  et  el.  1995). 

Managed  care  organizations  carefully  examine  all  aspects  of  heathcare: 
inpatient  costs,  outpatient  costs,  skilled  nursing  facility  costs,  home  care  costs,  durable 
medical  equipment  expense  and  even  hospice  costs.  Managed  care  focuses  on 
outpatient  preventive  healthcare  as  the  pivotal  point  in  the  provision  of  healthcare. 

The  managed  care  industry  believes  that  an  ounce  of  prevention  is  worth  more  than  a 
pound  of  cure,  their  financial  survival  is  dependent  upon  implementation  of  preventive 
health  care.  The  successful  managed  care  companies  educate  their  physician  partners 
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on  the  value  of  preventive  care  to  their  financial  success.  Managed  care,  or  more 
correctly,  managed  competition,  will  encompass  every  facet  of  healthcare  within  5 
years  (Stewart- Amidei  1994). 

Preventive  services  are  increasingly  viewed  as  the  fovmdation  of  quality 
healthcare,  and  are  primarily  delivered  in  the  outpatient/ambulatory  setting.  The 
quality  of  health  in  this  country  increasingly  depends  on  the  ambulatory  sector  for 
several  reasons:  1)  more  people  receive  medical  care  in  the  ambulatory  setting  than  the 
inpatient  setting;  2)  prompt  and  efficacious  treatment  of  an  episode  of  illness  may 
improve  the  patient’s  long-term  health  and;  3)  appropriate  outpatient  preventive 
interventions  have  significant  potential  to  reduce  total  healthcare  expenditures  by 
reducing  the  need  for  episodes  of  both  inpatient  and  outpatient  care.  Proper 
coordination  of  care  in  the  ambulatory  setting  can  reduce  the  costs  of  wasted  and 
redundant  work  and  the  need  to  repeat  poor-quality  work  (Lawthers,  et  el.  1993). 

Quality  healthcare  is  a  commodity  that  comes  with  a  high  price  tag. 

Consumers  ask  many  more  questions  about  the  healthcare  they  now  pay  more  and 
more  for  with  the  passage  of  each  year.  The  movements  toward  (1)  continuous  quality 
improvement  (CQI),  (2)  total  quality  management  (TQM),  and  (3)  similar  quality 
improvement  (QI)  approaches  are  seen  as  the  primary  means  of  managing  quality  in 
hospitals  and  other  healthcare  organizations.  These  approaches  reflect  values  and 
beliefs  about  appropriate  ways  to  manage  quality  (Sales,  et  el.  1995).  JCAHO’s 
approach  to  ensuring  quality,  which  emphasizes  functions  and  processes  to  improve 
quality  rather  than  specific  activities,  is  meant  to  promote  a  comprehensive,  integrated. 
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continuous  process.  Organizational  efforts  to  ensure  excellence  should  be  focused 
through  three  essential  managerial  processes:  (1)  defining  desired  performance  (setting 
standards),  (2)  measuring  performance,  and  (3)  maintaining  or  changing  performance 
(Penchansky  1993).  Confusion  exists  about  the  boundaries  between  QA,  UR,  UM,  and 
resource  management  (RM).  The  activities  of  these  processes  sometimes  overlap  and 
when  they  do,  their  purposes  and  methods  are  almost  indistinguishable.  The 
bovmdaries  separating  these  approaches  are  beginning  to  blur  (Flannery  1996, 
Penchansky  1993). 

Evaluation  of  quality  must  be  based  upon  objective  standards,  that  are 
objectively  measured.  The  introduction  of  the  Department  of  Defense’s  (DoD) 
TRICARE  Program  is  changing  the  way  healthcare  is  delivered  to  active  duty 
members,  their  family  members,  retired  service  members  and  their  family  members. 
TRICARE  is  the  DOD’s  medical  benefits  program  that  is  available  for  individuals 
eligible  for  CHAMPUS.  Each  encounter  with  the  healthcare  system  has  the  potential 
to  force  those  members  of  the  military  family  who  have  selected  a  civilian  gatekeeper 
to  pay  for  vdiat  was  once  looked  at  as  a  fi«e  “benefit  for  life.”  The  multiple  options 
available  under  TRICARE  are  drastically  changing  the  healthcare  options  available. 
The  question  has  to  be  asked  —  What  evaluation  measures  are  being  used  to  measxire 
the  quality  and  appropriateness  of  the  care  being  provided? 

The  attributes  of  the  delivery  of  healthcare  that  contribute  to  excellence  are:  (1) 
appropriateness  of  care,  (2)  coordination  of  that  care,  (3)  amelioration  of  the  problems, 
and  (4)  satisfaction  of  the  patient  (Penchansky  1993).  The  process  of  bringing  the 
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military  treatment  facility’s  (MTF)  and  the  contractor’s  UM  programs  into  alignment, 
as  TRICARE  attempts  to  do,  potentieilly  creates  problems  evaluating  the  care  being 
provided.  One  potential  contributor  to  consistent  and  successful  UM  is  the  use  of  the 
same  UM  guidelines.  Dr.  Joseph’s  directive  on  UM  guidelines  calls  for  all  MTFs  and 
TRICARE  contractors  to  use  INTERQUAL’s  UM  guidelines.  The  next  obvious 
question  -  Are  these  the  best  guidelines  available  for  use? 

a.  CONDITIONS  WfflCH  PROMPTED  THE  STUDY 

Military  treatment  facilities  (MTF)  and  TRICARE  contractors  are  currently 
directed  by  DOD  Health  Affairs  to  use  INTERQUAL’s  guidelines  for  their  UM 
programs  under  a  contract  with  INTERQUAL  for  a  three  year  period.  This  contract 
expires  in  1997.  INTERQUAL’s  UM  guidelines  are  not  the  only  guidelines  available 
for  use.  Is  this  contract  providing  the  best  buy  for  the  dollar?  The  question  under 
study  is  ~  What  criteria  should  be  used  to  identify  the  UM  guidelines  that  are  the  most 
appropriate  for  use  in  MTFs  and  by  TRICARE  contractors? 

b.  STATEMENT  OF  THE  QUESTION 

The  delivery  of  healthcare  is  not  an  exact  science,  it  is  an  art,  that  must  be 
practiced.  The  ongoing  practice  of  healthcare  delivery  will  reveal  that  different 
courses  of  treatment  will  eventually  lead  to  a  similar  favorable  outcome.  Providers  of 
healthcare  seek  to  find  the  optimal  course  of  care;  the  one  that  produces  the  favorable 
health  outcome  with  the  expenditure  of  the  least  amount  of  resources  (Doyle  &  Fenen 
1994).  Minimiang  the  cost  of  care  by  improving  delivery  efficiency  is  beneficial  to 
the  patient  and  all  payors.  The  ultimate  benefit  will  be  delivered  to  the  provider 
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because  their  capacity  to  deliver  care  increases  with  improved  efficiency.  The 
responsibility  of  healthcare  management  is  to  make  the  most  appropriate  use  of 
increasingly  limited  healthcare  resources.  Utilization  Management  plans  should  be 
developed  to  cover  the  entire  continuum  of  care,  from  the  first  treatment  encounter 
through  the  return  to  a  health  status  that  permits  a  return  to  “normal”  health  and 
activity  (Doyle  &  Ferren  1994). 

An  organized,  comprehensive  quality  management  (QM)  program  is  a 
hospital’s  response  to  social,  economic  and  legal  pressures.  QM  components  share 
four  critical  processes:  (1)  data  acquisition  and  management,  (2)  data  analysis,  (3)  data 
use,  and  (4)  oversight/supervision.  Organizational  strategies  accounting  for  all  four 
processes  are  essential  to  effective  quality  management/utilization  management 
programs  (Tennant  1994). 

INTERQUAL’s  review  system  is  a  clinically-based  system  designed  to  provide 
the  hospital  with  a  mechanism  to:  a)  identify  opportunities  to  affect  the  efiBcient  use  of 
resources  for  each  case,  and  b)  identify  variance  days  -  days  of  care  that  could  be 
delivered  in  a  less  costly  setting.  The  system  includes  utilization  review  criteria  and  a 
methodology  to  screen  for  the  medical  necessity  and  appropriateness  of  hospital 
admissions,  continued  hospitalization  and  services  provided  to  patients.  The  review 
system  is  designed  for  concurrent  and/or  retrospective  review.  The  criteria  do  not 
represent  physician  standards  of  care,  are  not  intended  for  use  in  reimbursement 
decisions,  and  are  not  designed  for  preadmission  authorization  decisions. 
INTERQUAL’s  criteria  address  medical  necessity  in  terms  of  tiie  severity  of  the 
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patient’s  condition,  the  intensity  of  treatment  provided  the  patient  and  whether 
discharge  can  be  safely  accomplished  (Tennant  1994). 

Milliman  &  Robertson  (M&R)  Inc.’s  (INTERQUAL’s  chief  competitor) 
Healthcare  Management  Guidelines  provide  a  quantitative  basis  for  both  on-line 
medical  management,  as  well  as  retrospective  analysis  and  assessment  of  healthcare 
management  programs  and  procedures.  The  guidelines  were  prepared  for  use  by 
“qualified”  medical  and  risk  management  professionals.  “The  guidelines  should  be 
applied  to  establish  a  course  of  treatment  for  a  specific  individual  only  in  conjunction 
with  the  application  of  professiraial  medical  judgement.”  (Doyle  &  Ferren  1994) 
Proper  application  of  M&R  guidelines  requires  consideration  of  all  factors  which 
might  affect  the  desired  or  actual  utilization  and  treatment  patterns. 

UM  is  viewed  as  one  of  fl»se  necessary  evils  healthcare  providers  must 
endme.  It  is  not  looked  upon  as  user  fiiendly,  and  some  doctors  will  tell  you  that  UR 
is  simply  a  way  to  identify  poor  utilizers.  UM  is  not  valued  by  providers  as  enhancing 
the  care  they  provide,  it  is  looked  iq)on  as  another  impediment  to  providing  care  to 
their  patients. 

The  selection  of  UM  guidelines  is  a  problem  the  DoD  will  fiice  in  1997.  How 
should  the  DoD  select  the  next  omtractor  for  UM  guidelines?  What  criteria  shotild  be 
used  to  select  UM  guidelines?  Should  the  contract  be  awarded  to  the  lowest  bidder,  or 
to  the  contractor  that  presents  die  best  product?  This  graduate  management  project 
was  tmdertaken  to  identify  a  set  of  criteria  that  can  be  applied  to  existing  UM 
guidelines  to  select  the  “best”  avaflable  for  use  throughout  the  DoD.  The  issue  of  cost 
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was  not  examined  as  a  part  of  this  project. 

c.  LITERATURE  REVIEW 

QUALITY  ASSURANCE 

Quality  Assurance  (QA)  is  the  formal  set  of  activities  used  to  review  and  affect 
the  quality  of  services  provided.  It  includes  quality  assessment  and  corrective  actions 
to  remedy  any  deficiencies  identified  in  the  quality  of  direct  patient,  administrative 
and  support  services.  Standard  QA  activities  alone  do  not  satisfy  the  needs  and 
requirements  of  health  plan  managers,  purchasers,  and  regulators  for  quality 
monitoring  and  management.  The  application  of  QA  by  providers  of  care  is  criticized 
for  excessive  focus  on  support  processes  and  structure  rather  than  actual  clinical 
processes.  Although  the  QA  model  does  not  explicitly  foster  it,  many  internal  and 
external  QA  programs  applied  to  providers  are  bad-apple  oriented,  with  a  focus  on 
finding  problems  and  someone  to  blame  (Panzer  1993).  QA  activities,  which 
primarily  focus  on  individual  physicians,  produce  two  negative  results:  (1)  potential 
problems  are  not  readily  generalized  to  the  delivery  system  as  a  whole  and  (2) 
physician  acceptance  is  jeopardized  because  deficiencies  in  care  are  attributed  to 
physicians  without  considering  the  roles  of  other  participants  in  the  health  delivery 
system  (Leatherman  1991). 

Currently,  there  is  no  thorough  on-going  program  to  ensure  public 
accountability  for  quality  of  care  in  all  hospitals  (Rubin  1994).  The  single-tracked 
nature  of  QA  programs  is  being  replaced  by  a  broader,  more  all  encompassing  process 
referred  to  as  Utilization  Management  (Henzler  1995).  Quality  of  care  is  the  degree  to 
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which  healthcare  services  for  individuals  and  populations  increase  the  likelihood  of 
desired  outcomes  and  are  commensurate  with  current  professional  knowledge 
(Goldfield  1991). 

All  customers  of  health  services,  including  enrollees  and  employers/payors, 
need  to  be  able  to  evaluate  and  understand  the  quality  measurement  tools  used  by 
healthcare  providers.  In  this  manner,  providers  will,  hopefully,  differentiate 
themselves  not  on  the  basis  of  a  proprietary  quality  measurement  system  but  rather  on 
tile  impact  these  programs  have  on  their  primary  customer:  their  patient  (Wise  1993). 
Cost  and  quality  are  now  related  in  the  minds  of  many  purchasers,  who  believe  that 
poor  quality  care  is  ultimately  more  expensive  than  good  quality  care.  Although  this 
conclusion  seems  obvious  and  rather  insignificant,  it  in  fact  represents  a  major  change 
in  attitude  from  the  long  held  assumptions  that  more  healthcare  and  more  expensive 
healthcare,  equaled  better  healthcare.  The  relationship  between  cost  and  quality  is 
captured  in  the  concept  of  value;  necessary  healthcare  at  a  reasonable  cost  (Panzer 
1993). 

.  “V 

Ifiil  possible  to  evaluate  the  structures,  processes,  and  outcomes  —  as  well  as 
the  improvements  ~  of  care  against  standards  that  define  the  quality  of  care 
(Schlackman  1991).  A  variety  of  applications  of  quality  information  directly  affect 
clinical  decision  making,  they  include:  critical  pathways,  case  maps,  algorithms,  and 
case  management  An  organized  effort  has  begun  to  design  in  a  standardized, 
preplanned  set  of  decisions,  as  well  as  information  feedback  to  clinicians  to  help  them 
fine  tune  their  decision  making  for  specified  patient  subgroups.  Benchmarking 
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appears  to  be  the  most  effective  technique  for  external  quality  review,  such  an  effort 
seeks  to  gather  information  about  the  outcomes  of  care  for  different  providers  and 
make  a  comparison.  The  QI  efforts  search  for  the  best  processes  by  looking  at  those 
with  the  best  outcomes,  these  are  cursory  beginnings  of  initiation  of  practice 
guidelines  (Panzer  1993). 

UTILIZATION  REVIEW 

Utilization  review  (UR)  programs  are  the  mechanism  healthcare  facilities  use 
to  demonstrate  appropriate  allocation  of  their  resources  (Lenox  1993).  The  scope  of 
UR  activities  has  evolved  from  initial  programs  directed  toward  the  avoidance  of 
Medicare  denials  and  timely  discharge  plaiming.  UR  has  been  a  retrospective  review 
of  the  care  and  services  provided  to  the  patient,  a  sort  of  post-care  look.  Initially 
adopted  as  a  weapon  to  attack  rising  health  plan  costs,  UR  has  grown  into  a  managed 
care  elder  statesman  that  gets  along  well  with  many  employer  groups/purchasers  and  is 
trying  to  make  peace  with  providers  (Grobman  1991).  UR  takes  place  at  the  level  of 
the  provider  organization  and  is  the  process  of  evaluating  the  efficiency  of  medical 
care,  based  on  examination  of  the  patient  record  (Torrens  1993,  Nelson  1983). 

One  purpose  of  UR  is  to  detect  inappropriate  medical  care  (i.e.,  care  provided 
in  an  unnecessarily  expensive  or  otherwise  resource  intensive  setting).  Inappropriate 
care  is  a  major  concern  for  hospitals,  since  reimbursement  may  be  fixed  and/or 
contested,  reduced,  or  denied  by  payers.  Inappropriate  care  mmecessarily  exposes 
patients  to  iatrogenic  conditions,  inconvenience,  and  financial  stress  (Nelson  1994). 

UR  policies  and  procedures  must  balance  the  organization’s  desire  to  ensure  that 
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services  are  provided  in  an  appropriate  and  cost-effective  manner  against  the 
possibility  that  unreasonable  or  inflexible  policies  and  procedures  may  be  alleged  to 
require  providers  to  withhold  medically  necessary  service  from  covered  individuals 
(Kongstevedt  1989).  UR  is  part  of  the  larger  effort  to  allocate  limited  resources  more 
efficiently  (Nelson  1994).  Employers  agree  that  UR’s  role  is  rapidly  changing  as 
employers  and  insurers  turn  to  more  aggressive  ways  of  controlling  utilization,  such  as 
managed  care  and  case  management  (Wise  1993). 

UR  has  been  required  for  participation  in  the  Medicare  program  since  the 
1960's  (Baschon  1992).  The  past  three  decades  have  seen  great  growth  in  efforts  by 
pm-chasers  of  healthcare  to  understand  and  influence  the  treatment  of  patients.  These 
efforts  reflect  purchasers’  concerns  that  their  increasing  expenditures  are  not  matched 
by  increasing  value  and  even  that  a  significant  amount  of  care  is  inappropriate  and 
wasteful  (Gray  1989).  Employers  contend  that  guiding  the  industry  toward  uniform 
procedures  is  an  insignificant  step  fiiat  doesn’t  address  the  real  problem  of  UR: 
documenting  its  effectiveness  (Wise  1993). 

Accreditation  standards,  first  released  by  the  Utilization  Review  Accreditation 
Committee  (URAC)  in  June  1991,  were  designed  to  encourage  consistency  in  die 
interactions  between  providers,  payers  and  consumers.  These  standards  ensure  that  the 
UR  process  causes  as  little  disruption  of  healthcare  delivery  as  possible;  sets  uniform 
national  standards  for  review  procedures,  turnaround  times  on  certification  and  non¬ 
certification  decisions  of  inpatient  stays;  and  provide  a  model  for  states  to  regulate  UR 
services.  Typically,  states  were  enacting  stafBng  requirements  to  ensure  that  board- 


12 


certified  physicians  were  overseeing  reviews,  and  that  UR  procedures  were 
documented  (Wise  1993).  Since  URAC  adopted  its  standards  in  1991, 10  states  have 
incorporated  the  same  language  or  waived  URAC  accredited  firms  for  compliance 
with  state  rules  (Wise  1993).  Efficiency  and  cost  containment  increasingly  are  seen  as 
complementary  to  healthcare  quality  (Hershey  1994).  Explicit  criteria  lead  to 
consistent  reviews  over  time  and  between  reviewers  (Lawthers,  et  el.  1993). 

DISCHARGE  PLANNING 

Discharge  Planning  (DP)  was  the  earliest  attempt  to  “manage”  a  resource 
intensive  patient,  and  was  the  first  attempt  to  coordinate  a  patient’s  hospital  stay.  The 
rationale  for  discharge  planning  is  twofold:  continuity  of  care  and  cost  effectiveness. 
DP  is  defined  as  the  process  of  activities  that  involve  the  patient  and  a  team  of 
individuals  fi'om  various  disciplines  working  together  to  effectively  move  that  patient 
fi-om  one  environment  to  another  (McKeehan  1981).  It  is  a  process  made  up  of  several 
steps/phases  whose  immediate  goal  is  to  anticipate  changes  in  patient  care  needs  and 
whose  long-term  goal  is  to  insure  continuity  of  healthcare  (Rorden  &  Taft  1990).  DP 
reduces  uncertainty  and  stress  during  a  hospital  stay  by  helping  patients  understand 
what  to  expect  and  reduces  the  possibility  of  fi:agmented  care  through  coordination  of 
services  (Rorden  &  Taft  1990). 

Every  caregiver  involved  with  a  patient  must  remain  alert  to  possible  discharge 
needs  throughout  the  acute  care  process.  DP  is  a  process  of  assessing  a  patients  needs 
and  obtaining  or  coordinating  appropriate  resources  as  they  move  throu^  the 
healthcare  system,  and  might  be  more  appropriately  called  continuity  of  care  planning 
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(O’Hare  &  Terry  1988).  The  discharge  plan  includes:  1)  evaluation  of  the  patient  for 
the  appropriateness  of  the  discharge;  2)  determination  of  the  optimal  site  of  care  and  of 
patient-care  resources;  and  3)  determination  that  financial  resources  are  adequate. 

DP  is  the  mechanism  that  guides  a  multi-disciplinary  effort  to  achieve  the 

successful  transfer  of  the  patient  from  the  healthcare  facility  to  an  alternate  site  of  care. 

The  complexity  of  the  plan  is  determined  by  the  patient's  medical  condition,  needs, 

and  goals.  DP,  (which  in  reality  should  b^in  prior  to  admission)  is  aimed  at  not  only 

meeting  the  needs  of  the  patient  virile  in  flie  hospital,  but  also  assuring  the  patient 

needs  will  be  met  after  discharge.  Meetmg  the  patient’s  need  could  involve;  (1) 

providing  assistance  with  financial  or  insurance  matters,  (2)  locating  necessary 

services/programs,  and  (3)  advocating  on  behalf  of  the  patients  and  families.  Not  all 

patients  admitted  to  the  hospital  require  the  services  of  the  discharge  planning  team. 

A  routine  uncomplicated  admission  usually  will  not  require  the  DP  team.  DP  efforts 

are  usually  directed  toward  those  resource  intensive  admissions,  or  the  fiequent 

readmissions  following  a  short  stay  at  home.  The  DP  team  becomes  involved  from 

& 

the  day  of  admission  to  assure  everythmg  is  in  place  to  allow  a  patient  to  be 
discharged  at  the  earliest  possible  time.  The  discharge  plan  should  be  reviewed, 
modified,  and  re-implemented  i^enever  die  patient  is  transferred  or  considered  for 
transfer  to  an  alternate  site.  A  disadvantage  associated  with  traditional  DP  is  that  it  is 
done  ivithout  the  advantage  of  a  case  manager,  “someone  in  charge”  to  see  that  things 
are  accomplished  in  a  timely  manner.  DP  was  the  first  evolutionary  step  into  the  case 
management  arena. 
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CASE  MANAGEMENT 


Case  management  has  captured  the  medical  community’s  attention  because  it 
supports  cost  effective,  quality  healthcare.  Case  management  based  on  a  utilization 
review  or  utilization  management  prototype  might  be  renamed  claims  management 
(Jennings  1994).  Case  management  is  a  boundary-spanning  approach;  instead  of 
providing  a  specific  direct  service,  it  utilizes  case  managers  who  link  the  client  to  the 
maze  of  direct  service  providers  (Sands,  et  el.  1994).  Case  management  as  a  system  is 
a  means  to  bring  managed  care  to  the  bedside,  and  as  a  component  of  the  managed 
care  system,  is  specifically  designed  to  coordinate  the  care  of  selected  patients.  It 
involves  the  clinical  management  of  targeted  patient  populations  from  admission  to 
discharge  (Smith,  et  el.  1994,  Jennings  1994).  Case-management  generally  is  regarded 
as  an  approach  to  client  management  that  endeavors  to  enhance  quality  of  life, 
decrease  fiiagmentation  of  services,  promote  self-care,  and  control  healthcare  costs 
(Benjamin  1988). 

Critical  pathways  are  important  tools  used  in  managed  care  and  case 
management.  Critical  pathways  are  a  timed  sequence  of  inti^entions  in  the  patient’s 
plan  of  care  to  achieve  desirable  outcomes  and  are  prepared  by  a  multi-  disciplinary 
group  of  clinicians  to  delineate  the  expected  process  of  care  delivery  (Jennings  1994). 
A  critical  pathway  is  a  written  plan  that  functions  as  a  guide  or  timetable  for  efficient 
and  timely  patient  care.  The  critical  pathway  acts  as  a  road  m^  to  direct  each 
discipline  on  the  essential  interventions  and  outcomes  that  must  be  accomplished  on  a 
given  day  or  within  a  given  time  finme  (Zander  1986).  These  interventions  are  done 
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by  an  interdisciplinary  team  of  registered  nurses,  licensed  practical  nurses,  primary 
care  physician  or  specialist,  nutritionist,  social  worker,  occupational  therapist,  physical 
therapist,  case  manger,  pharmacist,  home  health  nurse  and  other  professionals  as  well 
as  the  patient  and  his/her  immediate  family/significant  other.  Managed  Care 
Organizations  (MCOs)  want  to  see  critical  pathways  in  subacute  care  as  part  of  the 
credentialing  process.  MCOs  know  from  experience  that  critical  pathways  prevent 
unnecessary  utilization  of  services  and  provide  enormous  cost  savings  (Stahl  1995). 

The  goals  of  critical  pathways  are  to  achieve  measurable  quality  clinical 
outcomes  with  reasonable  financial  parameters  as  indicated  by  the  MCO  contract. 
Critical  pathways  provide  the  mechanism  to  develop  quality  clinical  outcomes  per 
Diagnosis  Related  Group  as  well  as  a  profit  and  loss  profile,  which  are  necessary  in 
strategic  planning  for  subacute  care  growth  (Stahl  1995).  Case  management  aims  to 
provide  consistency  of  care  based  on  measurable  outcomes  that  fulfill  a  standard  case 

management  plan  or  weU-negotiated  alternative  set  of  plans.  It  promises  to  decrease 

fragmentation  of  care  through  coUaborative  practice  patterns;  improve  quality  of 
services,  control  costs  by  reducing  wiited  time,  energy,  and  materials;  and  encourage 
a  timely  discharge  within  a  planned  length-of-stay  for  a  given  primary  diagnosis.  Its 
proponents  claim  that  it  enhances  communication  among  all  members  of  the 

healthcare  disciplines  and  promotes  professional  satisfaction  and  development  through 
increased  accountability  (Lynam  1994). 

Critical  pathways  are  mechanisms  that  facilitate  enactment  of  case 
management.  They  are  tools  to  map  or  track  a  patient’s  hospitalization,  articulating 
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critical  incidents  that  normally  take  place  on  a  given  hospital  day  and  include 
appropriate  interventions  to  achieve  certain  results.  In  most  models,  the  case  manager 
notes  and  records  deviations  from  the  critical  puth,  discusses  possible  causes  of 
deviations  with  appropriate  personnel,  and  arranges  for  remedial  action  to  be  taken 
(Lynam  1994).  One  major  barrier  to  achieving  quality  is  the  lack  of  ability  to  control 
the  care  delivery  process.  Case  management  is  one  way  of  overcoming  this  barrier 
(Dring,  et  el.  1994). 

The  case  manager  collaborates  on  providing  managed  care  for  clients  with 
catastrophic  health  problems  and  limited  resources.  The  innovative  management  of 
these  clients  can  reduce  costs  and  overall  length  of  stay  (Dring,  et  el.  1994). 

Regardless  of  the  quality  and  appropriateness  of  the  care  plan,  successful  case 
management  requires  sustained  and  continuous  monitoring  (Dring,  et  el.  1994).  Case 
management  focuses  on  the  relatively  few  beneficiaries  in  any  group  who  have 
generated  or  are  likely  to  generate  very  high  expenditures  (Gray  1989).  The 
intensified  involvement  of  government  and  private  purchasers  in  efforts  to  contain 
healthcare  costs  has  been  a  key  element  in  the  rise  of  more  direct  and  assertive 
programs  to  influence  patient  care  decisions.  Case  management  has  an  emphasis  on 
assessing  individual  needs  and  circumstances  and  then  plannings  arranging  and 
monitoring  needed  services.  There  seems  to  be  general  acceptance  that  case 
management  can  save  money  and  improve  the  quality  of  life  for  patients  and  families 
(Gray  1989). 

UTILIZATION  MANAGEMENT 
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Utilization  Management  (UM)  as  defined  by  the  American  Hospital 
Association  (AHA)  is  the  planning,  organizing,  directing,  and  controlling  of  the 
healthcare  product  in  a  cost-efficient  manner  while  maintaining  the  quality  of  patient 
care  and  contributing  to  the  overall  goals  of  the  institution.  Baschon  has  a  slightly 
different  definition:  managing  the  people,  services,  and  supplies  necessary  to  provide 
patient  care  (Baschon  1994).  The  AHA  definition  is  a  workable  definition  in  any 
healthcare  setting  since  it  speaks  in  very  general  terms  and  doesn’t  specify  how  it  will 
be  done.  The  AHA  has  simply  set  the  goal.  Gray  and  Field  define  UM  as  a  deliberate 
action  taken  by  purchasers  to  manage  healthcare  costs  and  influence  the  hospital’s 
and/or  physician’s  decision  making  in  order  to  increase  the  efficiency  and 
appropriateness  with  which  healthcare  services  are  provided. 

UM  is  part  of  a  complex  balancing  act  created  by  society’s  struggles  with  two 
important  questions.  First,  how  to  ensure  that  people  get  needed  medical  care  without 
spending  so  much  that  it  compromises  other  important  social  objectives?  Second,  how 
to  discourage  imnecessary  and  inappropriate  medical  services  without  jeopardizing 
necessary  high-quality  care?  UM  is  a  set  of  techniques  used  by  or  on  behalf  of 
purchasers  of  health  benefits  to  manage  healthcare  costs.  These  techniques  influence 
patient  care  decision-making  through  case-by-case  assessments  of  the  appropriateness 
of  care  prior  to  and  following  its  provision.  Little  evidence  exists  that  there  is  an 
impact  of  UM  methods  on  the  quality  of  care,  or  on  patient  and  provider  costs.  UM 
adds  to  the  administrative  burdens  of  practitioners  and  institutional  providers,  and 
contributes  to  feelings  of  reduced  professional  autonomy  and  satisfaction  (Gray  1989). 
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UM  is  a  program  that  involves  the  entire  healthcare  team.  Its  goal  is  to  provide  high 
quality  care  at  a  reasonable  cost  to  the  hospital  and  the  payor  (Baschon  1992). 

Utilization  management  programs  have  been  widely  used  to  control  hospital 
inpatient  costs,  but  little  is  known  about  their  potential  to  control  outpatient  costs 
(Wickizer  1995).  UM  is  designed  to  control  healthcare  costs  by  influencing  treatment 
decision  making  on  a  case-by-case  basis  through  coordination  of  care,  flexible 
allocation  of  treatment  and  financial  resources,  and  encouragement  of  less  restrictive 
treatment  settings  (Gustin  et  el.  1995).  UM  is  an  active  process  that  not  only  identifies 
inefficiencies,  but  attempts  to  eliminate  them  as  well  (Baschon  1992).  As  processes 
improve,  the  quality  improves  and  the  process  becomes  more  efficient  which 
conserves  resources  for  use  in  other  processes,  or  to  provide  appropriate  care  to  greater 
numbers  of  people. 

Physicians  are  responding  to  capitation  by  using  UM  techniques;  some  at  early 
stages  of  development,  that  were  previously  used  only  by  insurers.  The  physician- 
initiated  management  approach  represents  a  fundamental  transformation  in  the  practice 
of  medicine  (Kerr  et  el.  1995).  According  to  the  AHA’s  policy  statement:  healthcare 
organizations  should  be  accoimtable  for  delivering  high-quality  care  and  services; 
operating  an  internal  performance  improvement  system;  providing  useful  information 
to  purchasers  and  consumers  to  help  them  choose  providers,  plans  and  treatment 
options;  and  contributing  to  the  health  of  tibe  community  (Bergmann  1994). 

Third  party  payers  have  widely  publicized  the  cost  savings  they  have  achieved 
through  aggressive  UM  programs.  As  profit  margins  fi'om  Medicare  and  other  third 
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party  payers  continue  to  decline,  UM  takes  on  a  more  comprehensive  role.  In  addition 
to  ensuring  that  admissions  are  appropriate,  action  must  be  taken  to  ensure  that 
patients  stay  only  as  long  as  medically  necessary  to  recuperate  from  their  illnesses. 
Complications  must  be  minimized,  and  unrelated,  non-acute  problems  deferred  for 
treatment  to  the  outpatient  setting.  To  maximize  the  financial  benefits  of  utilization 
management  and  improve  patient  care,  resource  management  must  be  a  part  of  the  UM 
program.  This  is  not  to  say  that  length  of  stay  should  be  ignored.  Elimination  of 
medically  mmecessary  days  reduces  lengdi  of  stay  without  increasing  charges. 
Uimecessary  days  may  be  the  result  of  poor  scheduling,  inappropriate  test  sequencing, 
delays  in  initiation  of  discharge  planning.  UM  programs  should  be  designed  to 
identify  and  report  medically  uimecessary  days  and  patterns  of  inappropriate 
utilization.  The  UM  department,  through  it’s  access  to  vital  information,  has  many 
opportunities  to  influence  changes  in  clinical  practice  patterns.  This  information  must 
be  presented  to  administration  in  such  a  way  that  its  importance  is  immediately 
obvious.  Changing  inefficient  practices  can  dramatically  reduce  lost  or  denied 
charges,  and  greatly  improve  staff  productivity  (Baschon  1992). 

The  review  process  is  becoming  less  a  retrospective  review  of  whether  actions 
that  have  occurred  were  correct,  and  more  a  concurrent  and/or  prospective  review  of 
whether  actions  being  taken  or  planned  are  necessary  and  appropriate.  UM  is  a  part  of 
overall  peer  review  which  is  any  review  of  professional  medical  activity,  directly  or 
indirectly  related  to  quality  of  care.  Medial  record  review  is  used  to  monitor 
compliance  with  the  utilization  management  program,  and  is  the  primary  means  to 
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identify  utilization  problems. 

The  dominant  UM  strategy  is  prior  review  of  proposed  medical  services,  which 
includes  several  related  techniques.  Prior  review  provides  advance  evaluation  of 
whether  medical  services  proposed  for  a  specific  person  conform  to  provisions  of 
health  plans  that  limit  coverage  to  medically  necessary  care.  Most  prior  review 
programs  include  an  integrated  set  of  review  steps,  not  all  of  which  will  apply  to  any 
single  patient.  The  focus  may  be  on  the  site  of  care,  the  timing  or  duration  of  care,  or 
the  need  for  a  specific  procedure  or  other  service.  The  first  point  of  assessment,  often 
called  preadmission  review,  may  occur  before  an  elective  hospital  admission.  A 
patient  may  be  required  to  get  a  second  opinion  on  the  need  for  certain  proposed 
treatments  fi’om  a  practitioner  other  than  their  physician  (Gray  1989).  It  is  important 
to  note  that  prior  review  is  not  used  in  the  full  capitation  model.  The  fully  capitated 
provider  makes  all  decisions  regarding  medically  necessary  treatment. 

Dr.  Joseph  has  directed  that  DoD’s  UM  processes  (this  includes  TRICARE)  be 
(1)  patient  focused,  and  (2)  that  they  ensure  delivery  of  necessary  and  appropriate  care 

r 

at  the  most  cost  effective  level  without  jeopardizing  quality  or  access  (Joseph  1 994). 
UM  activities  and  information  are  essential  elements  in  the  overall  development  and 
incorporation  of  total  quality  improvement  activities  within  military  medicine.  The 
DoD’s  medical  budget  is  built  on  the  assumption  of  major  accomplishments  in  UM  in 
order  to  hold  the  atmual  rate  of  growth  to  levels  below  die  national  norm.  The  DOD 
UM  plan  calls  for  a  system  wliich  includes  prospective  review,  concurrent  review, 
discharge  planning,  case  management,  and  retrospective  review.  Retrospective  review 
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activities  are  especially  important,  since  they  provide  both  validation  of  review 
decisions  and  evaluation  of  overall  trends  in  utilization.  The  key  elements  of  an 
effective  UM  process  (UR,  case  management,  and  discharge  planning)  provide  a  basic 
foundation  for  evaluation  of  care  and  services,  and  development  of  best  clinical 
practices  such  as  practice  guidelines,  critical  pathways,  and  clinical  outcomes  studies 
(Joseph  1994). 

UR  and  UM  programs  are  most  effective  when  integrated  into  the  quality 
management  program,  and  when  they  create  their  desired  effect  using  resources  as 
economically  as  possible  (Tennant  1994).  However  as  with  any  system,  barriers  to 
UM  implementation  exist  and  must  be  addressed.  Some  common  barriers  include:  turf 
battles;  hospital  and  medical  staff  that  do  not  understand  the  payment  mechanism  has 
changed;  physician  and  staff  resistance  to  change;  a  system  that  is  unable  to  clearly 
identify  costs  associated  with  patient  care;  and  patients  that  fail  to  recognize  that 
healthcare  is  not  free.  Additional  barriers  are:  patients  and  family  demands  that  result 
from  unrealistic  expectations  of  the  healthcare  system;  frdlure  of  senior  management 
and  nursing  administration  to  actively  support  utilization  managenient;  failure  to 
educate  all  disciplines  about  their  role  in  utilization  and  resource  management  from 
the  start;  and  physician  practice  of  defensive  medicine.  The  final  set  of  barriers  are: 
physician  and  staff  failure  to  consider  cost  in  ordering  practices;  physicians  who  order 
tests  out  of  habit,  without  regard  to  new  technology;  peer  or  supervisor  pressure  on 
residents  and  staff  physicians  to  order  tests  or  keep  the  patient  hospitalized;  lack  of 
financial  incentive  for  physicians  to  encourage  proper  utilization  of  resource;  and  a 
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system  where  physician  reimbursement  may  encourage  longer  stays  and  over  use  of 
resources.  In  addition  to  these  barriers  are  the  four  common  mindsets  that  must  be 
overcome:  1)  “We  ve  always  done  it  that  way;”  2)  “That  won’t  work  in  our  hospital;” 
3)  “Our  physicians  won’t  do  that;”  and  4)  “Our  patients  are  sicker”  (Baschon  1994). 
The  implementation  of  a  UM  program,  or  the  change  to  a  new  UM  program 
encounters  many  barriers  and  problems  because  it  forces  people  out  of  their  comfort 
zone  and  into  unfamiliar  toritory.  A  change  to  a  new  UM  program  will  require 
involvement  fiom  the  top  levels  of  management  and  provider  staff,  and  an  intense 
education  program  for  all  staff  members. 

d.  PURPOSE 

The  purpose  of  this  graduate  management  project  is  to  identify  a  set  of  criteria 
to  be  used  in  the  selection  of  commercially  available  UM  guidelines  for  use  in  DOD 
healthcare.  These  criteria  will  be  developed  and  applied  to  two  commonly  used 
commercial  sets  of  guidelines  using  a  Delphi  process.  The  development  of  criteria  and 
selection  of  guidelines  will  be  based  on  die  decision  of  a  panel  of  UM  experts.  After 
this  set  of  criteria  for  selection  of  guidelines  is'identified  and  applied  to  the  two 
commercially  available  sets  of  guidelines,  a  recommendation  will  be  made  to  DoD 
Health  Affairs  regarding  the  set  of  guidelines  identified  as  the  most  appropriate  for  use 
in  MTFs.  The  recommendation  will  be  dependent  upon  the  panel  identifying  a  clear 
difference  between  the  sets  of  guidelines. 

2.  METHOD  AND  PROCEDURES 

THE  DELPHI  STUDY 
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The  Delphi  technique  was  developed  as  a  forecasting  tool  at  the  RAND 
Corporation  in  the  1940's.  The  technique  was  developed  as  an  attempt  to  deal  with 
future  decisions  by  making  systematic  use  of  the  intuitive  guesstimate  of  a  number  of 
experts.  The  RAND  Corporation  continued  to  refine  the  Delphi  procedure,  and  around 
1950  the  technique  was  used  on  problems  of  group  information  utilization.  It  has 
gamed  recognition  and  is  used  in  planning  settings  to  achieve  a  number  of  objectives: 
1)  Determine/Develop  a  range  of  possible  program  alternatives,  2)  Explore/Expose 
underlying  assumptions  or  information  leading  to  different  judgments,  3)  Seek  out 
information  which  may  generate  a  consensus  on  the  part  of  the  respondent  group,  4) 
Correlate  informed  judgments  on  a  topic  spanning  a  wide  range  of  disciplines,  and  5) 
Educate  the  respondent  group  as  to  the  diverse  and  interrelated  aspects  of  the  topic 
(Delbecq,  et  el.  1975). 

The  Delphi  technique  is  a  systematic,  iterative  method  of  forecasting  based  on 
independent  input  fi-om  a  group  of  experts.  The  objective  is  to  obtain  a  consensus  of 
opinion  firom  a  panel  of  experts  regarding  the  topic  under  research.  The  Delphi 
method  encourages  honert  opinion  wiiich  is  flee  fi-om  peer  group  pressure.  Because 
successive  rounds  of  information  reach  each  panel  member,  views  can  be  retracted, 
altered  or  added  with  the  benefit  of  considered  thought  (Williams  &  Webb  1994). 

Farrell  &  Scherer  (1983)  maintain  that  the  Delphi  technique  is  suitable  for  areas  where 
there  is  a  lack  of  empirical  data.  One  of  the  main  advantages  of  the  Delphi  approach  is 
Its  ability  to  guide  group  opinion  towards  a  final  decision  (McKenna  1994). 

CHARACTERISTICS  OF  THE  DELPHI  TECHNIQUE 
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The  Delphi  technique,  a  method  for  soliciting  and  combining  the  opinions  of 
experts,  involves  the  use  of  a  series  of  questionnaires  designed  to  produce  group 
consensus  and  eliminate  the  face-to-face  confrontation  experienced  on  panels  or 
committees.  Key  characteristics  of  the  Delphi  are: 

Anonymity:  The  obvious  advantage  of  guaranteed  anonymity  is  that  it 
encourages  opinions  which  are  not  influenced  by  peer  pressure  or  other  extrinsic 
factors.  It  avoids  certain  psychological  factors  present  in  fece-to-face  debate,  such  as 
undue  influence  by  a  dominant  individual  O^e  jury  room  effect),  the  bandwagon 
effect,  and  the  tmwillingness  to  abandon  a  position  once  it  has  been  publicly  stated 
(Wamick  &  Sullivan  1988).  It  can  also  lead  to  a  lack  of  accountability. 

Iteration  with  controlled  feedback:  This  is  achieved  in  Delphi  studies 
through  the  use  of  successive  questionnaires  which  seek  information  and/or  opinion 
from  the  respondents. 

Statistical  group  response:  This  builds  upon  the  information  fed  back  to  the 
Delphi  participants  within  the  successive  questionnaires.  It  provides  a  statistical 
of  the  group’s  views  on  specific  hems,  while  it  maintains  anonymity. 

The  use  of  experts:  The  originators  of  the  Delphi  technique  tend  not  to 
advocate  a  random  sample  of  panelists  who  are  representative  of  the  target  population. 
Instead,  the  use  of  experts  or  a  least  informed  advocates  is  recommended.  Consensus 
is  suggested  to  equate  with  51%  agreement  among  the  respondents  (Loughlin  1979, 
Goodman  1987). 

The  Delphi  technique  is  a  multistage  approach,  with  each  stage  building  on  the 
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results  of  the  previous  one  (McKenna  1994).  The  main  features  of  the  Delphi  survey 
are; 

1 .  It  is  a  postal  or  electronic  questionnaire:  (a)  this  preserves  independent 
thought  which  is  seen  as  important  to  prevent  individuals  being  unduly  influenced  by 
strong  personalities  in  the  group,  and  (b)  it  facilitates  participation  by  respondents 
from  a  wide  geographic  area. 

2.  It  is  a  non-random  sample  made  up  of  experts  in  the  field.  This  reflects  file 
aim  of  the  survey  which  is  to  produce  a  best-guess  consensus  on  a  question  of  policy 
in  an  area  v^diere  answers  are  required  but  unobtainable  through  more  empirically 
based  means.  The  use  of  experts  reflects  the  fact  that  it  is  not  an  opinion  poll  designed 
to  reflect  the  views  of  the  general  population.  Rather  it  is  an  opinion  poll  of  experts 
whose  views  might  not  be  shared  by  the  general  population  precisely  because  they  are 
less  well  informed. 

3.  It  consists  of  a  number  of  rounds  and  involves  feedback  to  respondents. 

This  reflects  the  aim  of  the  Delphi  survey  as  it  was  originally  conceived,  which  was  to 
elicit  a  consensus  of  opinion.  Results  of  each  round  are  therefore  analyzed  and  fed 
back  to  the  respondents  who  are  asked  to  reconsider  their  position  in  the  light  of  the 
overall  results.  Experts  who  continue  to  hold  deviant  views  may  be  asked  to  expand 
on  these  views  which  will  be  given  to  the  other  respondents  for  further  consideration. 
In  this  way  the  Delphi  survey  technique  aims  to  produce  a  consensus  from  the  experts 
on  the  topic  under  consideration  (Proctor  1994,  Whitman  1990,  Loughlin  1979). 

The  Delphi  Technique  has  been  applied  to  numerous  fields  since  its 
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development  in  the  1 950's.  In  the  past  few  years,  there  have  been  many  applications 
of  the  technique  in  such  areas  as  industry,  social/health  planning  at  the  community 
level,  evaluation  of  research  projects,  and  educational  innovations.  One  of  the  most 
extensive  uses  of  the  method  has  been  in  higher  learning.  In  education  and  healthcare 
the  focus  has  been  cost-effectiveness,  cost/benefit  analysis,  organizational  goals  and 
objectives,  and  consensus  rating  scales  and  values  (Lindeman  1975). 

Reasons  for  choosing  the  Delphi  technique  (Linstone  &  Turofif  1975) 

1.  The  research  problem  does  not  lend  itself  to  precise  analytical  techniques, 
but  can  benefit  from  subjective  judgements  on  a  collective  basis. 

2.  The  research  population  may  present  diverse  backgrounds  with  respect  to 
experience  or  expertise. 

3.  Time,  cost,  and  logistics  would  make  frequent  meetings  of  all  the  subjects 
unfeasible. 

As  the  literature  points  out,  the  Delphi  Technique  has  been  used  for  nearly  50 
years  in  fields  ranging  from  industry,  community  planning,  education,  and  healthcare. 
Due  to  its  long  history  of  use  and  success,  the  process  is  widely  accepted  for  accuracy, 
reliability,  and  validity.  The  technique  can  be  used  in  most  types  of  decision  making 
processes;  from  the  most  complicated  survey  to  the  simplest  interview.  The  Delphi 
technique  is  recognized  as  being  cost-effective  (Lyons  1981,  Polit  &  Hungler  1987). 
Sackman  (1975)  who  has  written  an  exhaustive  critique  on  the  subject  admits  how 
inexpensive  it  is  to  use,  while  Reid  (1988)  maintains  that  it  is  one  of  the  cheapest 
research  methods  available.  A  form  of  the  Delphi  Technique  is  used  in  our  everyday 
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lives;  whether  it  s  buying  a  car,  computer,  house,  or  even  groceries;  we  seek  out  who 
we  think  are  the  area  experts. 

STUDY  DESIGN 

In  this  study  a  sample  of  UM  experts  was  identified  fi-om  two  distinct 
populations.  The  first  group  came  fi'om  UM  experts  working  in  various  DoD  MTFs, 
Lead  Agents,  or  Health  Service  Support  Area  assignments.  The  second  group  came 
fi'om  UM  experts  working  in  San  Antonio  based  HMOs.  An  invitation  to  participate 
(See  Annex  A)  was  sent  to  each  identified  UM  expert.  An  identified  UM  expert  was 
operationally  defined  as  an  individual  whose  primary  job  was  UM,  and  v^iio  had  been 
performing  UM  for  at  least  last  twelve  months.  An  additional  requirement  to 
participate  in  this  study  asked  each  participant  to  have  a  working  knowledge  of  the 
UM  guidelines  conunercially  prepared  by:  (1)  INTERQUAL  and  (2)  M&R. 

The  first  round  of  the  Delphi  study  asked  the  panel  of  experts  to  identify  no 
more  than  seven  criteria  they  would  use  to  make  their  personal  selection  of 
commercially  available  UM  guidelines.  (See  appendix  B  for  first  round  questionnaire) 
During  the  first  round  panelists  were  asked  to  submit  demographic  data  about:  a) 
profession,  b)  number  of  years  working  in  that  profession,  c)  educational  level,  d) 
number  of  years  working  in  UM,  e)  current  employer,  f)  number  of  years  with  current 
employer,  g)  age,  and  h)  gender.  (See  Appendix  C)  In  the  first  round  thirty  different 
criteria  were  identified  by  the  panel  of  twelve  experts.  Criteria  that  were  similar  were 
combined  or  reworded  to  maintain  the  context  of  the  authors’  intent.  (See  Appendix 
D) 
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In  round  two  of  the  study,  the  experts  were  asked  to  review  the  list  of  thirty 
criteria  generated  in  round  one,  and  select  a  minimum  of  seven  criteria  they  considered 
valid  for  use  in  the  selection  of  UM  guidelines.  Each  participant  was  asked  to  mark 
each  criteria  yes  if  they  would  use  the  criteria  for  selection,  and  no  if  they  would  not. 

(See  rotmd  two  results  at  Appendix  E-1)  Consensus  criteria  for  use  in  round  three 
were  those  criteria  that  fifty-one  percent  of  the  panel  marked  yes  they  considered  the 
criteria  were  valid  for  guideline  selection.  (See  column  marked  %  of  group  agreement 
at  Appendix  E-1) 

In  round  three  of  the  study,  the  experts  were  asked  to  indicate  how  strongly 
they  agreed  or  disagreed  that  the  consensus  criteria  fi’om  round  two  were  met  first  by 
INTERQUAL  and  then  by  M&R  UM  guidelines.  (  See  Appendixes  F  and  G  for  third 
round  questioimaire) 

3.  THE  RESULTS 

Each  participant  was  mailed,  hand-delivered,  or  e-mailed  qtiestionnaires  for  round  one  of 
the  Delphi  study.  Each  participant  received  a  demogrrqrhic  questionnaire  with  the  round  one 
criteria  questionnaire.  The  demognq)hics  revealed  the  following  (see  Appendix  C): 

(1)  The  group  consisted  of  ten  Registered  Nurses,  one  Physical  Ther^ist,  and  one  Health 
Care  Administrator. 

(2)  The  total  number  of  professional  years  was  two  hundred  twenty  years  with  an  average 
of  18.3  years. 

(3)  Six  participants  hold  Masters  Degrees,  five  have  Bachelors  Degrees,  and  one  is  a 
Diploma  graduate  RN  currently  working  toward  her  Bachelors  Degree. 


29 


(4)  Tlie  total  years  experience  in  utilization  management  for  the  group  was  seventy^  eight 
years,  with  an  average  of  six  and  a  half  years. 

(5)  The  expert  panel  consisted  of  four  active  duty  Army  officers,  one  active  duty  Air 
Force  officer,  one  Department  of  the  Army  Civilian,  five  PacifiCare  employees,  and  one  Humana 
employee.  The  total  number  of  years  employed  was  1 14,  with  an  average  length  of  employment 
of  nine  and  a  half  years. 

(6)  The  panel  was  made  up  of  two  males  and  ten  females,  with  an  average  age  of  41 .2 

years. 

Round  one  produced  thirty  different  criteria,  after  similar/identical  criteria  were  combined 
or  reworded.  The  intent  of  the  original  criteria  was  maintained  feir  each  criteria  combined  or 
reworded.  If  the  original  intent  could  not  be  maintained  during  combination  wMi  a  similar 
criteria,  the  original  criteria  was  retained,  (see  Appendix  D) 

In  roimd  two,  the  panel  of  experts  was  asked  to  review  the  list  of  thirty  criteria  generated 
by  the  group.  Criteria  participants  considered  valid  for  die  selection  of  commercially  prepared 
UM  guidelines  were  marked  with  a  yes.  Criteria  participants  did  not  consider  valid  for  selection 
of  commercially  prepared  UM  guideline  were  marked  with  a  no.  The  results  are  displayed  af^'V 
Appendix  E- 1 .  Eighteen  of  the  thirty  criteria  identified  in  round  one  received  a  consensus 
(simple  majority,  7  of  12  SMEs)  vote  as  valid  for  the  selection  of  commercially  prepared  UM 
guidelines.  (See  shaded  criteria  Appendix  E-1) 

In  round  three  the  panel  of  experts  was  asked  to  rate  INTERQUAL’s  and  M&R’s 
guidelines  against  the  eighteen  consensus  criteria.  (See  Appendixes  F  &  G)  Appendixes  H-1  and 
I-l  display  the  results  of  the  ratings  each  set  of  guidelines  received.  The  participants  rated  their 
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opinion  whether  the  guideline  met  the  criteria  on  a  scale  of  strongly  agree,  agree,  no  opinion, 
disagree,  and  strongly  disagree.  The  responses  have  been  assigned  the  following  numerical 
values:  a)  strongly  agree  -  {5},  b)  agree  -  {3},  c)  no  opinion  -  {0},  d)  disagree  -  {-3},  and  e) 
strongly  disagree  --  {-5}. 

Only  upon  receipt  of  third  round  input  was  it  discovered  several  members  of  the  expat 
panel  had  a  misimderstanding  of  the  concept  of  “working  knowledge”  of  the  guidelines.  Five 
members  of  the  panel  did  not  have  the  required  working  knowledge  of  both  sets  of  guidelines 
(see  Appendix  H-1  &  I-l).  Failure  to  identify  this  flaw  in  SME  selection  earlier  in  the  process 
made  recruitment  of  new  panel  membos  unrealistic  in  terms  of  timely  completion  of  the  project. 
The  change  in  number  of  panel  membos  folly  capable  to  participate  in  the  study  resulted  in  the 
need  to  reevaluate  the  results  of  round  two  (See  Appendix  E-2). 

4.  DISCUSSION 

The  individuals  selected  for  participation  in  this  study  were  all  very  eager  to  express  foeir 
thoughts  on  the  UM  criteria,  as  was  evidenced  by  the  rapid  tum-around  time  on  the  first  two 
rounds  of  the  study.  The  first  two  rounds  were  completed  in  toi  days.  Each  round  of  the  sbufy 
had  been  allotted  ten  days.  The  rapid  tum-around  time  may  be  attributed  to  the  virtual 
information  world  we  are  developing  through  the  use  of  electronic  mail  and  fax  machines  fwfiie 
exchange  of  files  and  mfoimation.  The  virtually  instantaneous  communication  possible  through 
die  electronic  medixun  has  eliminated  foe  long  delays  experienced  by  those  who  formerly  relied 
on  the  US  mail  system  for  dissemination  of  their  study  topics,  questionnaires,  and  the  return  of 
responses. 

The  return  of  the  third  round  questionnaire  brought  with  it  problems.  1)  Only  three  of  foe 
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twelve  participants  returned  the  questionnaires  within  the  agreed  upon  time  frame  often  days. 
The  other  nine  questionnaires  were  returned  as  much  as  ten  days  late.  2)  Five  of  the  twelve 
panelists  annotated  on  the  bottom  of  the  questionnaires  they  were  not  familiar  enough  with  one 
or  the  other  set  of  guidelines  to  rate  both  sets  of  guidelines.  These  five  panelists  did  rate  the  set 
of  guidelines  they  were  currently  using.  Three  of  the  seven  civilian  experts  stated  they  were  not 
familiar  enough  with  the  INTERQUAL  guidelines  to  rate  them,  and  two  of  the  five  military 
experts  stated  they  were  not  familiar  oiough  with  the  M&R  guidelines  to  rate  them. 

The  inability  of  five  panelists  to  participate  fully  in  the  study  created  a  quandary  about 
how  to  complete  the  study.  The  consmsus  criteria  were  reevaluated  to  identify  criteria  that 
would  be  selected  by  the  smaller  panel  of  experts.  The  reevaluation  showed  that  only  sixteen 
criteria  were  selected  by  the  smaller  panel.  Various  options  were  available,  considered,  and 
examined  to  identify  their  effects  on  die  results.  The  following  was  observed. 

The  first  option  considered  was  to  simply  tabulate  the  scores  as  they  were  returned, 
adding  2eros  for  those  experts  unfamiliar  with  a  set  of  guidelines.  As  depicted  in  Appendixes  H- 
1,  and  I-l,  INTERQUAL’s  cumulative  score  was  299  with  an  average  of  33.2,  and  M&R’s 
cumulative  score  was  492  widi  an  average  score  of '49.2;  using  the  eighteen  criteria  identified  by 
the  entire  panel  of  twelve  ejq)erts.  Appendixes  H-2,  and  1-2  depict  INTERQUAL’S  cumulative 
score  decreased  to  253  with  an  averse  of  28.8,  and  M&R’s  cumulative  score  decreased  to  454 
with  an  average  of  45.4;  using  the  sixteen  criteria  identified  by  the  panel  of  seven  fully  qualified 
subject  matter  experts.  This  option  was  not  appropriate  since  the  INTERQUAL  score  was  based 
on  nine  respondents  and  the  M&  R  score  was  based  on  ten  respondents. 

The  next  option  considered  was  to  employ  a  statistical  technique  of  averaging  the 
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responses  for  each  criteria,  and  substituting  that  average  for  the  missing  data.  The  average  is  a 
more  valid  number  to  use  than  zero  and  will  skew  the  data  less  (Finstuen  1995).  Appendixes  J-1 
and  K-1  display  the  results  of  averaging  the  responses  for  each  of  the  eighteen  criteria  and 
substituting  the  average.  INTERQUAL’s  adjusted  score  rose  to  a  total  of  398.7  with  an  average 
score  of  33.2  and  M&R’s  adjusted  total  score  rose  to  590.4  with  an  average  score  of  49.2. 
Appendixes  J-2  and  K-2  display  the  results  of  averaging  the  responses  for  the  sixteen  criteria 
identified  by  the  panel  of  seven  members.  INTERQUAL’S  adjusted  score  fell  to  345.3  with  an 
average  score  of  28.8  and  M&R’s  adjusted  total  score  fell  to  544.8  with  an  average  score  of  45.4. 

The  final  option  considered  was  to  eliminate  those  respondents  unable  to  evaluate  both 
sets  of  guidelines  fi'om  the  study.  This  option  resulted  in  the  use  of  seven  experts’  evaluations  of 
sixteen  criteria.  Appendixes  L  and  M  display  the  data.  INTERQUAL’s  total  score  dropped  to 
173  with  an  average  score  of  24.7,  and  M&R’s  total  score  dropped  to  357  with  an  average  score 
of51. 

An  examination  of  the  scores  achieved  by  the  sets  of  guidelines  evaliiated  by  each  option 

shows: 

1)  The  first  technique  is'bbviously  flawed  and  would  not  produce  results  that  could  be 
siq^rted.  Appendixes  H-1  and  I-l  based  on  eighteen  criteria  demonstrate  a  difference  between 
guideline  total  scores  of  193  points,  and  a  difference  between  average  scores  of  16.  Appendixes 
H-2  &  1-2  which  are  based  on  sixteen  criteria  identified  by  the  seven  member  panel  show  a 
difference  of  195  points,  and  a  difference  between  averages  scores  of  16.6. 

2)  The  use  of  the  statistical  method  of  averaging  is  an  improvement  over  the  first 
technique,  with  a  difference  between  guideline  total  scores  of  191 .7  points,  with  a  difference 
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between  average  scores  of  16  using  the  eighteen  criteria  identified  in  round  two  (See  Appendixes 
J-1  and  K-1).  The  total  score  difference  using  sixteen  criteria  was  199.5  with  a  difference 
between  average  scores  of  16.6  (See  appendixes  J-2  &  K-2).  The  second  technique,  although  an 
improvement  over  the  first,  is  not  acceptable  because  it  uses  scores  generated  by  individuals  not 
fully  qualified  for  participation  in  the  study. 

3)  Using  the  third  technique  and  eliminating  respondents  not  familiar  with  both  sets  of 
guidelines,  results  in  a  difference  between  guideline  total  scores  of  184  points,  and  the  differaice 
between  average  scores  of  26.3.  (See  total  score  and  average  score  boxes  at  Appendix  L&M) 

The  first  two  options  were  rejected,  due  to  the  inaccuracy  of  the  results  produced.  The 
first  two  options  were  inaccurate  because  five  individuals  not  fully  qualified  to  participate  in  the 
study  supplied  data  that  was  included  in  the  analysis.  The  third  method  of  analysis  was  selected 
for  use  in  drawing  conclusions  since  h  is  not  biased  by  either  missing  data,  substituted  data,  or 
participants  not  fully  qualified  to  take  part  in  the  process.  The  expert  panel  size  was  substantially 
reduced;  however  the  reduction  in  potential  bias  will  facilitate  a  better  decision-making  process. 
Had  time  allowed,  a  group  of  substitute  panelists  ^o  met  all  criteria  for  participation  would 
have  beeh*  assembled.  A  problem  in  the  design  of  the  invitation  to  prospective  subject  matta 
experts  was  the  inaccurate  definition  and/or  interpretation  of  the  term  “working  knowledge”  of 
both  sets  of  guidelines.  A  better  definition  would  have  been  the  requirement  to  have  “worked 
with”  both  sets  of  guidelines.  A  “working  knowledge”  of  the  guidelines  as  interpreted  by  tiie 
SME  invitees  at  tiie  begirming  of  the  study  did  not  translate  to  a  sufficient  understanding  of  the 
guidelines  for  full  participation  in  the  study. 

The  overall  analysis  of  the  semes  given  by  the  final  expert  panel  of  seven  members 
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shows  that  M&R’s  guidelines  received  an  average  score  of  51  and  INTERQUAL’s  guidelines 
received  an  average  score  of  24.7.  (See  average  score  box  Appendix  L&M)  A  criteria-by-criteria 
analysis  of  the  results  reveals  the  expert  panel  gave  higher  scores  for  agreement  that  the  selection 
criteria  were  met  by  M«&R’s  guidelines  in  Hof  16  consensus  criteria.  The  first  of  the  two 
selection  criteria  where  M&R  received  a  lower  score  than  INTERQUAL  was  being  “nationally 
recognized/utilized”;  M&R’s  composite  score  was  28  and  INTERQUAL’s  was  33.  (See  shaded 
row  Appendixes  L&M)  The  second  exception  was  the  criteria  that  said  the  “guidelines  should  be 
valid  and  reliable,  objective,  non-controversial,  and  clinically  sound”;  M&R’s  composite  score 
was  17  and  INTERQUAL’s  score  was  19.  (See  shaded  row  Appendixes  L&M) 

5.  CONCLUSIONS 

The  reduction  in  the  size  of  the  expert  panel  reduces  the  impact  of  the  findings,  but  does 
not  negate  the  findings.  This  expert  panel  agreed  M&R’s  UM  guidelines  met  the  set  of  selection 
criteria  better  than  INTERQUAL  s.  This  is  evidenced  by  the  higher  composite  and  average 
scores.  As  evidenced  by  Appendix  N,  regardless  of  the  number  of  SMEs  (12  vs  7)  included  in 
the  analysis  or  number  of  consensus  criteria  (18  vs  16)  M&R  consistently  and  substantially  was 
found  the  superior  tool.  The  results  can  be  interpreted  that  the  expert  panel  agreed  M&R’s  UM 
guidelines  met  the  consensus  criteria  to  a  higher  degree  than  INTERQUAL’s.  Therefore  the 
expert  panel  would  choose  M&R’s  guidelines  over  INTERQUAL’s.  This  decision  has  not 
considered  cost  since  cost  was  not  evaluated  as  a  part  of  this  study. 

6.  RECOMMENDATIONS 

1)  DoD  Health  Affairs  should  use  the  research  methodology  used  in  this  study  for  the 
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selection  of  UM  guidelines  for  the  next  contract.  The  methodology  should  be  duplicated  with  a 
larger  group  of  subject  matter  experts  to  further  validate  the  appropriateness  of  these  selection 
criteria. 

2)  Research  should  be  conducted  into  the  cost  of  contracting  with  both  M&R  and 
INTERQUAL  for  use  of  their  UM  guidelines.  A  cost  benefit  analysis  should  be  done,  that  must 
include  the  training  that  would  be  required  for  the  change  across  the  military  health  care  system. 
The  analysis  must  also  take  into  accoimt  any  change  in  the  way  care  is  provided,  any  resulting 
change  in  length  of  stay,  and  any  change  in  care  outcomes. 

3)  Milliman  and  Robertson’s  UM  guidelines  should  be  given  strong  consideration  as  the 
UM  guidelines  of  choice  for  the  next  contract  period. 


SPECIAL  NOTE;  The  opinions  expressed  here  are  those  of  the  authors  alone,  and  do  not 
represent  those  of  the  U.S.  Army  Medical  Department,  the  U.S.  Army,  or  the  Department  of 
Defense. 
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25  March  1996 


My  name  is  LTC  William  Wyckoff.  I  am  currently  completing  the 
residency  phase  of  the  U.S.  Army-Baylor  Masters  in  Healthcare  Administration.  I 
have  chosen  for  my  Graduate  Management  Project,  (a  requirement  for  graduation) 
an  exploration  of  expert  opinions  of  Utilization  Management  (UM)  guidelines.  I 
have  selected  this  project  in  the  hopes  of  making  a  recommendation  to  the 
Department  of  Defense,  Office  of  Health  Affairs,  about  the  UM  guidelines  that 
should  be  selected  for  the  contract  to  be  signed  in  1997,  based  on  the  knowledge 
and  experience  of  UM  experts. 

I  am  enlisting  your  assistance  as  an  “expert”  in  UM  to  cwnplete  a  Delphi 
Process.  You  have  been  identified  as  a  knowledgeable  individual  in  the  UM 
process.  I  am  asking  for  your  assistance  in  completing  the  tasL  I  do  not  anticipate 
the  process  taking  more  than  3-4  rounds  to  complete.  I  will  have  the  initial 
questionnaire  to  you  by  Wednesday  of  this  week,  and  ask  that  you  reply  within 
seven  days.  Based  on  your  responses,  I  will  send  the  second  questionnaire  to  you 
within  three  days.  Once  consensus  (51%  of  the  participants)  is  reached,  the  final 
questioimaire  will  be  sent  you.  This  is  not  a  labor  intensive  effort  on  your  part,  but 
your  knowledge  and  experience  will  prove  invaluable  to  the  research  effort. 

The  requirements  to  participate  are:  1)  willingness,  2)  familiarity  wifii  bodi 
Interqual  and  Milliman  &  Robertson  guidelines,  3)  a  minimum  of 30-45  minutfM;  to 
devote  to  each  round  of  the  process. 

I  will  send  the  questionnaire  for  each  round  to  those  not  located  in  San 
Antonio  via  Email  or  Fax,  and  will  hand  deliver  those  in  the  San  Antonio  area. 
Anonymity  of  responses  will  be  guaranteed. 

Please  respond  either  by  telephone  (210)  979-2304,  Fax  (210)  979-2406,  or 
Email  your  availability  or  non-availability  to  participate.  If  you  have  any 
questions  that  I  have  not  addressed,  please  feel  fi-ee  to  contact  me. 


APPENDIX  A 
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Round  #  1  Questionnaire 


List  no  more  than  seven  (7)  criteria  you  would  use  in  the  selection  of  UM 
guidelines.  Please  disregard  any  current  contracts  or  obligations  that 
may  exist  within  your  organization.  Please  be  specific  rather  than 
general. 


1. 
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APPENDIX  B 


Participant  Demographics 
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Appendix 


The  following  is  a  list  of  30  criteria  that  have  been  recommended  for 
the  selection  of  UM  Guidelines.  Similar  criteria  have  been 
combined/ reworded  to  include  the  essence  of  the  suggested  criteria. 
Thank  you  for  your  input.  Please  indicate  whether  you  would  use 
each  of  the  criteria  to  select  UM  guidelines  by  writing  yes  or  no 
beside  each  criteria.  Please  select  at  least  seven  criteria. 

Criteria 

1.  Nationally  recognized/utilized  criteria,  must  have  credentials  as  a  tool 
-  i.e.  AMA,  NCQA,  JCAHO  recognition. 

2.  The  guidelines  should  have  broad  acceptance  and  demonstrated 
efficacy  when  used  in  civilian  managed  care  or^mizations,  research  based. 

3.  Do  they  show  ways  to  minimize  the  cost  of  health  care  without,  taking 
away  quality  of  health  care,  enhances  appropriate  resource  use. 

4.  The  guidelines  should  be  valid  and  reliable,  objective,  non- 
controversial,  clinically  sound. 

5.  Physician  affiliation  preference/  must  have  physician  buy  in 

6.  Can  they  be  integrated  effectively  with  Health  Care  Provider  support 

7 .  Guidelines  provide  reasonable  approach  to  implementing  health  care 
management 

8.  Adaptability  to  local  market/  physician  practice  patterns/  flexible  - 
adaptable  -  cookbooks  are  “guidelines”  not  absolutes,  provider  oriented 

9.  Relatively  easy  to  use  by  health  carfe  professionals/  easy  to 
imderstand,  user  friendly,  with  clear,  concise,  unambiguous  criteria. 

10.  Readily  available/  on-line  in  all  data  reports  on  UR,  Ideally, 
guideline  software  would  be  in  a  relational  database  format  with 
standardized  reports  and  easy  ad  hoc  capabilities. 

11.  Guidelines  avmlable  for  medical,  surgical,  pediatric,  mental  health, 
obstetrics.  The  guidelines(s)  must  evaluate  critical  aspects  of  care  in 
adult  and  pediatric  age  groups,  medical  surgical  and  mental  health,  and 
inpatient  and  outpatient  care. 

12.  Do  they  cover  the  entire  period  of  care  from  first  encounter  to  last, 
proactive  planning,  not  reactive. 

APPENDIX  D 
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13.  Are  they  consistent  with  established  data  that  is  relevant  and  current 

14.  Do  they  consist  of  current  and  cost-effective  protocols,  the  guidelines 
should  have  regular  updates  based  on  continuous  quality  improvement 
of  the  instrument  itself,  and  changes  in  healthcare  delivery. 

15.  Do  they  have  the  ability  to  educate  and  guide  health  provider,  health 
plan,  and  member 

16.  Do  they  confirm  to  clinical  competency 

17.  There  should  be  hard  copies  and  software  of  the  guidelines  for 
use  on  lap  tops. 

18.  Meets  Quality  Standards  -  meets  professionally  recognized  standards 
of  care. 

19.  Able  to  track  out  of  the  ordinary  problems  -  to  identify  problems  & 
potential  problems. 

20.  Involves  all  disciplines 

21.  Staff  must  have  tredning  -  med  techs,  nurses,  data  people 

22.  Prospective  Review  -  Identification  of  cases  for  case  management  or 
special  resource  needs  and  understanding  of  demand  management 

23.  Concurrent  Review  -  Identification  for  discharge  planning 
requirements.  Identification  of  facility  process  for  improvement 

24.  Retrospective  Review,  Focused  Studies  -  Identification  of  need  for 
intervention  to  improve  facilify  process.  -  Identify  requirements  in 
infrastructure  needs 

25.  Case  Management,  Discharge  planning  -  Patient  advocate,  social 
work  model  not  just  a  steward  of  CHAMPUS  dollars 

26.  Patient  Profiling  for  clinical  conditions  (morbidity  etc.)  use  of 
resources  -  Without  this  capability  UM  is  very  difficult  to  achieve 

27 .  Data  base  application  is  part  of  package  and  is  easily 
interpreted/utilized 
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28.  Provider  Profiling,  style  of  practice  -  The  other  part  of  health  care 
delivery.  The  other  part  is  the  patient.  Without  this  part  how  can  any 
management  take  place. 

29.  Demand  Management,  Disease  prevention,  epidemiology  acuity  and 
comorbidity  -  This  the  neglected  part  of  UM  but  requires  patient 
profiling,  intake  assessment  to  PCM  etc.  This  is  really  what  we  are 
buying  health  status 

30.  Highlights  where  patient  should  best  be  located  &  not  detailed 
clinical  criteria  as  primary  focus 
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ROUND  2  RESULTS  USING  TWELVE  PANELISTS 


APPENDIX 


•ROUND  2  RESULTS  USING  THE  FINAL  SEVEN  PANELISTS 


The  table^low  displays  the  results  of  the  expert  panel  selection  of  criteria  for  round  three. 
YES=1  NO  =  0  - - - - 


NOTE:  -  r  Shaded  area  represents  criteria  that  were  not  included  in  the  final  analysis 
when  the  panel  size  was  decreased  to  seven. 
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APPENDIX  E-2 


ROUND  3  INTERQUAL  QUESTIONNAIRE 
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ROUND  3  M  R  QUESTIONNAIRE 
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ROUND  3  INTERQUAL  RESULTS  ADJUSTED  USING  16  CRITERIA 
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Indicates  averages  based  on  ten  respondents 
:  Indicates  disqualifbd  as  an  8ME. 
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P  Indicates  averages  based  on  ten  respondents 
Indicates  disquallfed  as  an  8ME. 
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NOTE  -  •  SHADED  CRITERIA  REFER  TO  COMMENTS  ON  PAGE  35. 


AVERAGE  SCORE  COMPARISON 


INTERQUAL 


ROUND  3  USING  18  CRITERIA 

33.2 

ROUND  3  USING  12  CRITERIA 

28.8 

ROUND  3  ADJUSTED  USING  18  CRITERIA 

33.2 

ROUND  3  ADJUSTED  USING  16  CRITERIA 

28.8 

ROUND  3  USING  7  PANELIST  &  16  CRITERIA 

24.7 

M&R 


49.2 

45.4 
49.2 

45.5 
51 
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